Semi-Lazy Learning: Combining Clustering and Classifiers to Build More Accurate Models

Eager learners such as neural networks, decision trees, and naïve Bayes classifiers construct a single model from the training data before observing any test set instances. In contrast, lazy learners such as Knearest neighbor consider a test set instance before they generalize beyond the training data. This allows making predictions from only a specific selection of instances most similar to the test set instance which has been shown to outperform eager learners on a number of problems. However, lazy learners require the storing and querying of the entire training data set for each instance which is unsuitable for the large amounts of data typically found in many applications. We introduce and illustrate the benefits of an example of semi-lazy learning that combines clustering and classification models and has the benefits of eager and lazy learners. We propose dividing the instances using cluster-based segmentation and then using an eager learner to build a classification model for each cluster. This has the effect of dividing the instance space into a number of distinct regions and building a local model for each. Our experiments on UCI data sets show clustering the data into segments then building classification models using a variety of eager learners for each segment often results in a greater overall cross-validated accuracy than building a single model or using a pure lazy approach such as K-Nearest-Neighbor. We can also consider our approach to semi-lazy learning as an example of the divide and conquer (DAC) strategy used in many scientific fields to divide a complex problem into a set of simpler problems. Finally, we find that the misclassified instances are more likely to be outliers with respect to the clustering segmentation.